6S: Distributing Crawling and Searching Across Web Peers
نویسندگان
چکیده
A collaborative peer network application called 6Search (6S) is proposed to address the scalability limitations of centralized search engines. 6S peers depend on a local adaptive routing algorithm to dynamically change the topology of the peer network and search for the best neighbors to answer their queries. We validate prototypes of the 6S network via simulations with 70− 500 model users based on actual Web crawls and find that the network topology rapidly converges from a random network to a small world network, with clusters emerging from user communities with shared interests. We finally compare the quality of the results with those obtained by centralized search engines such as Google, suggesting that 6S can draw advantages from the context and coverage of the peer collective.
منابع مشابه
6S: A Collaborative Web Search Network
6S is a collaborative peer network application, aimed to extend the current model of centralized search engines with large numbers of autonomous, distributed, micro-search engines. Each peer within the 6S network crawls the Web in a focused way, guided by its user’s information context. This way better contextual coverage can be achieved. Each peer also acts within the network by submitting, fo...
متن کاملCrawling and Searching the Hidden Web
OF THE DISSERTATION Crawling and Searching the Hidden Web
متن کاملTopic-Driven Crawlers: Machine Learning Issues
Topic driven crawlers are increasingly seen as a way to address the scalability limitations of universal search engines, by distributing the crawling process across users, queries, or even client computers. The context available to such crawlers can guide the navigation of links with the goal of efficiently locating highly relevant target pages. We developed a framework to fairly evaluate topic...
متن کاملIPTV-RM: A Resources Monitoring Architecture for P2P IPTV Systems
Resources monitoring is an important problem of the overall efficient usage and control of P2P IPTV systems. The resources of IPTV can include all distributing servers, programs and peers. Several researches have tried to address this issue, but most of them illuminated P2P traffic characterization, identification and user behavior. The main contributions of this paper are twofold. Firstly, a r...
متن کاملSemantic Overlay Networks for Peer-to-peer Web Search
We consider a network of peers, where each peer has its own collection obtained by individually crawling the web. When designing a distributed search system for such networks, an important task is how to efficiently perform query routing, i.e., how to find the most promising peers to answer the query. However, the efficiency of those routing techniques depends heavily on the underlying network ...
متن کامل